1. Field of the Invention
The present invention relates generally to neural networks of the type generally suitable for pattern recognition, robotic control and optical correlation; and, more particularly, to individual processing elements that comprise such neural networks.
2. Description of the Related Art
Synaptic coupling between neurons in biological neural networks gives rise to variable states of neuron activity resulting in some being turned on (firing), some turned off (not firing), and others in transition. These transitions occur in a nonlinear fashion described by a sigmoid transfer curve, which is shown in FIG. 1(a). That figure shows a plot of mean neuron firing output versus input potential to the neuron (membrane potential).
In the Hopfield electrical model of a biological neural network described in "Neural Networks and Physical Systems with Emergent Collective Computational Abilities," Proc. Nat. Acad. Sci. U.S.A., Vol. 79 (1982) pp. 2554-2558, and in "Computing with Neural Circuits: A Model," Science 233, (1986) pp. 625-633, each neuron is represented by an amplifier which has a sigmoid voltage transfer curve (see FIG. 1(b)). For this model, the cutoff or negative output of the amplifier is analogous to the neuron not firing, with full turn on of the amplifier corresponding to maximum firing rate.
While this model has proven to be quite effective in approximating the biological neuron response, the voltage conditions of the model remove temporal dependence from the model.